Victim Caching with Better Block Replacement

Cache memories help bridge the cycle-time gap between fast microprocessors and relatively slow main memories. By holding recently referenced regions of memory, caches can reduce the number of cycles the processor must stall while waiting for data. As the disparity between processor and main memory cycle times increases by 40 percent or more per year cache performance becomes ever more critical. Caches exhibit one of three types of misses: compulsory, capacity, and conflict. Previous studies have shown that conflict misses account for roughly 20% to 40% of all misses. Victim caches have been proposed as one cost-effective method for significantly reducing the number of conflict misses incurred. In this paper, we propose a new victim cache block replacement algorithm which improves upon the effectiveness of victim caches in reducing conflict misses. The obvious block replacement algorithm to use in a victim cache is the least recently used (LRU) algorithm; however, no time stamp information between sets of the main cache is currently available, so an LRU block replacement algorithm in the victim cache cannot be implemented. Instead, LRU is approximated by using the time that a block was evicted from the main cache, i.e., a least recently evicted (LRE) algorithm. We first demonstrate that victim cache performance improves when a non-practical true LRU block replacement algorithm is used rather than the LRE block replacement algorithm. We then propose a new, practically implementable block replacement scheme that more closely approximates LRU than LRE does, and we show that it closes most of the performance gap between LRU and LRE.